Relative 3-D reconstruction using multiple uncalibrated images

In this paper, we show how relative SD reconstruction for point correspondences of multiple images f r o m uncalibrated cameras can be achieved through reference points. The original contributions with respect t o other related works in the field are mainly a direct global method for relative SD reconstruction, a geometrical method t o select a correct set of reference points among all image points. Ezperimental resulis f rom both simulated and real image sequences are presented. 1 Relative positioning From a single image, no depth can be computed without a priori information. Even more, no invariant can be computed from a general set of points [3]. This problem becomes feasible using multiple images. The process is composed of two major steps. First image features are matched in the different images. Then, from such a correspondence, depth is easily computed using standard triangulation. This approach suffers from <several drawbacks: firstly the calibration process is an error sensitive process; secondly it cannot always be performed on line, particularly when the imaging system is obtained by a dynamic system with zooming, focusing and moving. Similarly stereo vision with a moving camera is impossible as the standard tool for locating the position of a camera with translation and rotation does not reach the required precision for calibrating such a multistereo system. Introducing in each image beacons with exact known position may overcome these drawbacks: calibration and reconstruction are then solved in the same process [2, 11. But for many problems it is impossible to provide such carefully positioned reference points. The alternative approach is to use points in the scene as reference frame without knowing their coordinates nor the camera parameters. This has been investigated by several researchers these past few years, for instance in [lo, 11, 9, 17, 8, 14, 161. This year, three independent teams approached the same problem of 3D reconstruction from uncalibrated cameras, and all three with the same projective basis. Faugeras [4] published an insightful algebraic method to do 3D projective reconstruction. He demonstrated that once the epipolar geometry is somehow determined, 3D projective structure can be reconstructed up to a collineation by assigning 5 reference points to the standard projective basis. One month later, Hartley published his paper [7] in which he described the similar approach in a slightly different way. The same month appeared our technical report [12] performed differently with our first experimental results. The original contributions of this paper are mainly twofold. First, we describe a direct 3D relative reconstruction method, which differs from that of Faugeras and Hartley in that our method is formulated globally as a least squares estimation method which does not need to first estimate the epipolar geometry, and also the method makes full use of redundancy of multiple images. Secondly, we provide a geometrical way to choose among the set of points those which can be selected as reference points. The selected reference points should not be degenerated, i.e. no four of them coplanar. This result allows to derive a computational way to choose the correct reference points. We assume that the reader is familiar with elementary projective geometry, as it can be found in the first chapters of [15] (see also [5]). 2 Using scene reference points This section provides the basic equations of 3D reconstruction problem, together with the self calibration problem. This derivation was developed independently from these recently published by Faugeras in [4]. The basic starting point is similar to this work, however the way to solve it was influenced by the way photogrammetrists simultaneously calibrate their camera and reconstruct the scene, by using carefully located beacons (cf. [l]).